Podcast: Play in new window | Download

Subscribe: RSS

Date: July 22nd, 2019

Reference: Yadav et al. A Multifaceted Intervention Improves Prescribing for Acute Respiratory Infection for Adults and Children in Emergency Department and Urgent Care Settings. AEM July 2019

Guest Skeptic: Dr. Chris Bond is an emergency medicine physician and clinical lecturer in Calgary. He is also an avid FOAM supporter/producer through various online outlets including TheSGEM.

Case: A 25-year-old female presents to the urgent care with two days of cough, purulent sputum, fever and myalgias. Vitals signs are within normal limits and her exam is unremarkable. She asks for a prescription for antibiotics to help treat her infection. 

Background: Inappropriate antibiotic use exposes patients to opportunistic infections, accelerates the development of antibiotic resistant bacteria and leads to adverse drug events [1]. Acute respiratory infections (ARIs) are a major cause of unnecessary antibiotic use. Emergency departments (EDs) in the United States write 10 million antibiotic prescriptions each year, approximately half of which are inappropriate [2, 3, 4]. Given these risks, strategies to reduce inappropriate antibiotic use in the ED and urgent care centers (UCCs) are needed.

Despite recognizing the need for antibiotic stewardship by EDs and emergency providers, this has not led to practice change [5, 6]. Providers in the ED and UCC setting are faced with numerous challenges that may limit change, including: Frequent interruptions, boarding and overcrowding, frequent patient handoffs, and the need to see high volumes of patients [7, 8, 9].

There is evidence in both the medical literature and economic theory to support using a package of feedback, nudges and peer comparisons to improve prescribing outcomes. This has been shown to reduce unnecessary antibiotic prescribing in primary care, and in one study of peer comparisons in outpatient clinics and doctor’s offices, these improvements were sustained for at least 12 months after the interventions were completed [10, 11, 12].

Richard Thaler and Cass Sunstein wrote a book on the nudge theory. The book is called Nudge: Improving Decisions about Health, Wealth, and Happiness. The authors discuss psychologic and behavioral economics research to support active engineering of choice architecture. It’s a great book to put on your reading list.

Clinical Question: Is an enhanced intervention using audit and feedback, peer comparisons, and nudges more effective than a standard intervention in reducing inappropriate antibiotic prescribing for acute respiratory infections by clinicians in an ED/UCC setting?

Reference: Yadav et al. A Multifaceted Intervention Improves Prescribing for Acute Respiratory Infection for Adults and Children in Emergency Department and Urgent Care Settings. AEM July 2019

  • Population: Clinicians (general ED physicians, pediatric ED physicians, advanced care practitioners, internists and pediatricians) at five EDs and four UCCs in three academic health systems who prescribed antibiotics for ARIs.
    • Excluded: Resident physicians
  • Intervention: Enhanced intervention: This used all the elements of the adapted intervention, but also included peer comparison feedback via email, comparison to top performing peers, and additional locally tailored public posters demonstrating commitment to judicious antibiotic use.
  • Comparison: Adapted intervention: This incorporated strategies from the Centre for Disease Control and Prevention’s Core Elements for Outpatient Antibiotic Stewardship, including provider and patient education, a physician champion and departmental feedback. This used adapted brochures and other campaign messages for acute care providers.
  • Outcome:
    • Primary Outcome: Rate of inappropriate outpatient antibiotic prescribing for acute respiratory infections diagnosis that were deemed antibiotic-nonresponsive.
    • Secondary Outcome: Difference between the enhanced and adapted intervention groups in antibiotic prescribing.

This is an SGEMHOP episode which means we have the lead author on the show. Dr. Kabir Yadav is an Associate Professor and the Vice Chair for Academic Affairs at Harbor-UCLA Medical Center.

Dr. Larissa May

We also have the senior author on this HOP publication, Dr. Larissa May. She is a Professor of Emergency Medicine at the University of California Davis and Directs the UC Davis Health Emergency Department and Outpatient Antibiotic Stewardship Program.

Authors’ Conclusions: “Implementation of antibiotic stewardship for ARI is feasible and effective in the ED and UCC settings. More intensive behavioral nudging methods were not more effective in high-performance settings.”

Quality Checklist for Randomized Clinical Trials:

  1. The study population included or focused on those in the emergency department. Yes
  2. The physicians were adequately randomized. Yes
  3. The randomization process was concealed. Yes
  4. The physicians were analyzed in the groups to which they were randomized. Yes
  5. The study physicians were recruited consecutively (i.e. no selection bias). Yes
  6. The clinicians in both groups were similar with respect to prognostic factors. Unsure
  7. All participants (clinicians, outcome assessors) were unaware of group allocation. No
  8. All groups were treated equally except for the intervention. Yes
  9. Follow-up was complete (i.e. at least 80% for both groups). Yes
  10. All physician/provider-important outcomes were considered. Yes
  11. The treatment effect was large enough and precise enough to be clinically significant. Yes

Key Results: They identified 44,820 ARI visits to the emergency department or Urgent Care Center among 292 clinicians across the nine sites.

Both adaptive and enhanced interventions worked to reduce inappropriate antibiotic prescribing viral acute respiratory illnesses.

  • Primary Outcome: Inappropriate antibiotic prescribing for ARI
    • Decrease from 6.2% (95%CI 4.5%-7.9%) to 2.4% (95%CI 1.3% -3.4%)
    • After adjusting for provider, seasonal and institutional fixed effects, there was a significant year over year reduction from baseline to intervention period (OR 0.67, 95% CI 0.54-0.82) with an absolute effect size of 0.7% (0.2-1.2%)

The baseline antibiotic prescribing rate for antibiotic-inappropriate ARIs during flu season of 2016-2017 was 4.3% across all sites but varied between 2.1-7.4% depending on site.

  • Secondary Outcomes: Difference between the enhanced and adapted intervention groups in antibiotic prescribing.
    • There was a non-significant (p=0.06) difference in differences between the reduction in inappropriate antibiotic prescribing between the enhance and adapted groups.

1. Cluster Randomization: You selected a cluster-randomized design for this trial which can decrease the power and the precision of the study. Why did you select this method of randomization? Why not just randomize all the clinicians to adapted or enhanced intervention?

A key challenge to a practice change intervention is contamination, wherein individual providers randomized to different arms may influence each other in unpredictable ways. To address this, we chose to randomize each physically distinct study site to one study arm, with the goal of minimizing providers in different study arms influencing each other.

2. Lack of Control Group: How can we really conclude this intervention resulted in a reduction in antibiotic prescribing without a control group where there was no intervention at all? Inappropriate antibiotic prescribing for ARI could be going done because of external factors beyond the adapted and enhanced intervention.

Lack of a contemporaneous controls is a valid concern. Given the participating institutions had incentives to rapidly deploy antibiotic stewardship, it was impossible to get buy-in to be a control site for the duration of the study. While we did look back at the prior year’s data to look for seasonally-adjusted trends, contemporaneous influences could not be easily accounted for. We could have designed a stepped-wedge cluster randomized design such that each site gets the intervention in a prescribed order, and sites not yet receiving the intervention act as contemporaneous controls. This is the design we are using for an ongoing scale-and-spread study currently underway.

3. Wrong Sites: Were you studying the wrong EDs and UCCs? These sites performed extremely well at baseline with very low inappropriate prescribing rates (2.1-7.4%). Should you be looking at community hospitals and UCCs not associated with academic centers?

This was surprising to us as well. According to National Quality Forum acute bronchitis quality metric used for pay-for-performance at two of the participating sites, inappropriate prescribing was in the 60-70% range, justifying a need for stewardship.  It turns out that it may be the metric we used, which we believe is closer to the true rate of inappropriate prescribing rate (conservative), may be driving it down. We did note that the pediatric sites were low prescribers overall, and that the adult urgent care site in Los Angeles County did start much higher.  National data from children’s hospital EDs suggests very low rate of prescribing at 2.5%.

4. Hawthorne Effect: This study is at significant risk of both a Hawthorne effect and altered coding of discharge diagnoses (eg. saying more pneumonias rather than upper respiratory track infections and then giving antibiotics). How can this risk be mitigated?

Insofar as the Hawthorne effect is considered to be the self-corrective behavior of participants when they know they are being observed; one could argue this is actually part any antibiotic stewardship intervention! I think the real question is that is such an effect sustainable, especially when providers are inundated with quality measure after quality measure that they are supposed to pay “special attention” to?

5. ICD-10 Coding: Has the method of identifying antibiotic-nonresponsive ARI diagnoses with ICD-10 been validated to be accurate?

We adapted the outcome measure used in the Meeker et al study done in primary care settings to the ED/UCC setting. That schema was based on ICD-9 codes, as was the National Quality Forum acute bronchitis metric. Other studies such as by Gerber et al have used their own codes for an outcome. Our outcome was based on a complete review of the ICD-10 codebook through consensus of the physician investigators and reviewed by the Centre for Disease Control and Prevention. It is publicly available on the MITIGATE toolkit for people to review and refine if they feel it is necessary. While it may not be perfect, we do believe it is a conservative outcome of inappropriate antibiotic prescribing that would be acceptable to providers receiving feedback. As such, it may not compare to other measures such as Choosing Wisely that may show higher rates as they include things that can be potentially appropriate to treatment with antibiotics like acute sinusitis. Limitations of ICD codes etc.—only as good as being coded.

6. Contamination: Some clinicians worked at multiple sites but were assigned to the intervention of the site where they spent at least 80% of their time. This threshold was lowered to >50% at the six CHCO sites. Would this not contaminate the results and make them more difficult to interpret?

Unlike the participating adult and mixed populations sites, the Children’s Hospital of Colorado sites often had providers that worked at more than one site, which potentially explains both the downward trend overall and potentially smaller effect size of their sites. However, as noted before, pediatric sites had lower prescribing from the outset, matching national trends. The potential for contamination is well taken, however, and to address it, a subgroup analysis simulating bounds of contamination effects could be undertaken as part of a battery of post hoc hypothesis-generating subgroup analyses consolidated in a manuscript that explores secondary analyses of the data.

7. Variety of Clinicians: You had a variety of clinicians providing care. This included general EM physicians, pediatric EM physicians,advanced care practitioners, internists and pediatricians). Did you perform any subgroup analyses for hypothesis generating purposes?

As noted above, we are considering a number of secondary analysis in a post hoc manner for exploratory purposes. Unfortunately, we are limited in the ability to analyze individual provider type in this study as we were prohibited from collecting provider demographic data by the Internal Review Board (IRB). We do, however, intend to analyze the performance of sites when sub-grouped by type of site. Moreover, follow-up studies at other clinical sites will collect demographic data, so hopefully this question can be addressed then.

8. Demographics: The IRB did not allow you to collect demographics on the clinicians. Were you interested in whether or not the different interventions were more or less effective based on gender, age or years of practice?

Prior studies on knowledge translation have suggested that there may be differences in uptake of new evidence based on demographic differences.  Initially more focused on age/years of practice but now a more critical eye toward gender. We do intend to explore these differences in follow-up studies where we collect demographic data and clinician type and years of practice.

9. Feedback Nudge: How positive was the feedback of “top performer vs. not top-performer” It seems the email just says: “you are not a top performer”. Would a more encouraging message be more helpful? What about listing the top performers at each physicians site? Could there be the opposite effect where the person would take pride in being the worst (Bart Simpson – Underachiever and proud of it man)

I felt the same way. Jason Doctor, an expert in cognitive psychology, suggests that the worst reaction is indifference, stating “upsetting and motivating are not mutually exclusive”. This wording is meant to challenge their self image as a top performer, and immediately follow-up with how they can improve. We also developed frequently asked questions (FAQs) meant to be transparent and objective about how we determined the outcome, and what was needed to be a top performer. It is important to note that everyone could become a top performer. There was little pushback and hurt feelings and mostly humorous responses.

10. What’s the Right Amount: Is getting to zero inappropriate antibiotic use a realistic goal? Would we not be at risk of causing more harm at that point by missed prescriptions in serious bacterial illnesses that should get an antibiotic? We are not perfect diagnosticians. What is the right amount of inappropriate antibiotic prescribing?

I think yes, assuming metric is true and coding is correct. Many of our physicians had rates of zero. It may be truer for some conditions such as nonspecific URI and acute bronchitis vs pharyngitis (where the outcome specification may not be able to parse out the viral pharyngitis from bacterial).

Those are the ten nerdy questions. Is there anything else you want to say about your SGEM Hot Off the Press publication?

Dr. Yadav

This project was guided by implementation science, which is a rigorous, theory-driven approach to practice improvement. It relies on careful deliberation of the local conditions when preparing for an intervention (which may modify what you do), followed by a mixed-methods approach to conducting the intervention and measuring several outcomes related to implementation processes. I think it holds great promise for elevated local quality improvement projects to interventions worthy of knowledge translation. Chris Carpenter, who the SGEM listeners may be familiar with, is a national/international expert on implementation science and has helped lead several initiatives on both the conduct and reporting of implementation science projects.

There is one more thing I would like to ask you about. Your study was featured on the Skeptics Guide to the Universe Podcast (Episode#728). They used your publication in their Science or Fiction section. One thing that bothered me was they representation that your intervention decreased inappropriate antibiotic prescribing by over 30%. This information seems to have come from a press release. While there was an odds ratio of 0.67 from baseline the absolute effect size was a reduction of 0.7%. I found the press release claiming the intervention reduced the overuse of antibiotics by one-third misleading and wondered if you would comment.

We absolutely agree with you that absolute effects rather than relative effects are the preferred way to report scientific findings. It is, however, challenging to make scientific findings accessible an interesting to the general public, and university media relations would, like the press in general, like to draw the reader to click on the article. The impact of science shouldn’t be guided by press release, Tweet or even this podcast. On a related note, there have been interesting articles written recently about this around the reporting of use of Vitamin C for sepsis. You should be skeptical of press releases and always go to the original study.

Comment on Authors’ Conclusion Compared to SGEM Conclusion: We agree that implementation of strategies to reduced inappropriate antibiotic prescribing for acute respiratory infections is feasible and likely effective.

SGEM Bottom Line: Consider implementing strategies to reduce inappropriate antibiotic prescribing in your ED or UCC.

Case Resolution: After completing your history and physical examination, you conclude that this patient has a viral illness and do not prescribe an antibiotic.

Clinical Application: This study provides strategies that could be tried to reduce unnecessary antibiotics for acute respiratory infections in the ED and UCC.

Dr. Chris Bond

What Do I Tell My Patient? Your history, reassuring vital signs and examination do not show any evidence of an infection requiring antibiotics. If anything, they may lead to harm, such as diarrhea, stomach upset, rashes and even nasty intestinal infections are possible. At this point you are likely to improve with fluids, rest and ibuprofen/acetaminophen for your fever and muscle aches. If you are developing significantly worsening shortness of breath or your fever is persistent after another few days, you should be re-assessed.

Keener Kontest: There were multiple winners last episode when I presented with Swami live at the NY ACEP meeting. Omadacycline belongs to the tetracycline class of antibiotic drugs.

Listen to the podcast to hear this weeks’ trivia question. If you know the answer, send an email to TheSGEM@gmail.com with “keener” in the subject line. The first correct answer will receive a cool skeptical prize.

SGEMHOP: Now it is your turn SGEMers. What do you think of this episode on EMTALA violations in psychiatric emergencies? Tweet your comments using #SGEMHOP. What questions do you have for Larissa and Kabir and theirteam? Ask them on the SGEM blog. The best social media feedback will be published in AEM.

Also, don’t forget those of you who are subscribers to Academic Emergency Medicine can head over to the AEM home page to get CME credit for this podcast and article. We will put the process on the SGEM blog:

  • Go to the Wiley Health Learning website
  • Register and create a log in
  • Search for Academic Emergency Medicine – “July”
  • Complete the five questions and submit your answers
  • Please email Corey (coreyheitzmd@gmail.com) with any questions or difficulties.

Remember to be skeptical of anything you learn, even if you heard it on the Skeptics’ Guide to Emergency Medicine.

References:

  1. Presidential Advisory Council on Combating Antibiotic- resistant Bacteria. National Action Plan for Combating Antibiotic-resistant Bacteria. Washington (DC): The White House, 2015.
  2. Donnelly JP, Baddley JW, Wang HE. Antibiotic utilization for acute respiratory tract infections in U.S. emergency departments. Antimicrob Agents Chemother 2014;58: 1451–7.
  3. Centers for Disease Control and Prevention. FastStats: Emergency Department Visits. Available from: http:// www.cdc.gov/nchs/fastats/emergency-department.htm. Accessed October 30, 2015.
  4. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Preva- lence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA 2016;315: 1864–73.
  5. May L, Gudger G, Armstrong P, et al. Multisite explo- ration of clinical decision making for antibiotic use by emergency medicine providers using quantitative and qual- itative methods. Infect Control Hosp Epidemiol 2014;35:1114–25.
  6. Linder JA, Doctor JN, Friedberg MW, et al. Time of day and the decision to prescribe antibiotics. JAMA Intern Med 2014;174:2029–31.
  7. Chisholm CD, Collison EK, Nelson DR, Cordell WH. Emergency department workplace interruptions: are emer- gency physicians “interrupt-driven” and “multitasking”? Acad Emerg Med 2000;7:1239–43.
  8. Bernstein SL, Bernstein E, Boudreaux ED, et al. Public health considerations in knowledge translation in the emergency department. Acad Emerg Med 2007;14:1036– 41.
  9. Schafermeyer RW, Asplin BR. Hospital and emergency department crowding in the United States. Emerg Med Australas 2003;15:22–7.
  10. Persell SD, Doctor JN, Friedberg MW, et al. Behavioral interventions to reduce inappropriate antibiotic prescrib- ing: a randomized pilot trial. BMC Infect Dis 2016;16: 373.
  11. Meeker D, Linder JA, Fox CR, et al. Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA 2016;315:562–70.
  12. Linder JA, Meeker D, Fox CR, et al. Effects of behavioral interventions on inappropriate antibiotic prescribing in primary care 12 months after stopping interventions. JAMA 2017;318:1391–2.